Scaling Up Influence Functions

نویسندگان

چکیده

We address efficient calculation of influence functions for tracking predictions back to the training data. propose and analyze a new approach speeding up inverse Hessian based on Arnoldi iteration. With this improvement, we achieve, best our knowledge, first successful implementation that scales full-size (language vision) Transformer models with several hundreds millions parameters. evaluate in image classification sequence-to-sequence tasks tens hundred examples. Our code is available at https://github.com/google-research/jax-influence.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Rod influence on hue-scaling functions

Rod influence on hue appearance of spectral lights was characterized by comparing the scaling of red, green, yellow, and blue hue sensations for an 8 degrees-diameter, 7 degrees-eccentric test spot under conditions that minimized (cone plateau) and maximized (dark adapted) rod influence at two mesopic light levels (1.5 and 3.0 log scoptic trolands). At the lower light level, the hue-scaling fun...

متن کامل

The up-scaling of ecosystem functions in a heterogeneous world

Earth is in the midst of a biodiversity crisis that is impacting the functioning of ecosystems and the delivery of valued goods and services. However, the implications of large scale species losses are often inferred from small scale ecosystem functioning experiments with little knowledge of how the dominant drivers of functioning shift across scales. Here, by integrating observational and mani...

متن کامل

Exploiting submodular value functions for scaling up active perception

In active perception tasks, an agent aims to select sensory actions that reduce its uncertainty about one or more hidden variables. For example, a mobile robot takes sensory actions to efficiently navigate in a new environment. While partially observable Markov decision processes (POMDPs) provide a natural model for such problems, reward functions that directly penalize uncertainty in the agent...

متن کامل

Scaling Up

Most analytical and experimental results on evolutionary programming (EP) are obtained using low-dimensional problems, e.g., smaller than 50. It is unclear, however, whether the empirical results obtained from the low-dimensional problems still hold for high-dimensional cases. This paper investigates the behaviour of four diierent EP algorithms for large-scale problems, i.e., problems whose dim...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence

سال: 2022

ISSN: ['2159-5399', '2374-3468']

DOI: https://doi.org/10.1609/aaai.v36i8.20791